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ABSTRACT 

Quantitative assessment instruments have been 
developed to identify those skills or behaviors which individuals 
with developmental disabilities need in order to become competitively 
employed, and to measure individuals' progress in the program and 
hence measure program effectiveness. Such assessment instruments are 
problematic, as each employment situation uniquely requires its ov;n 
particular combination of skills. An attempt to apply quantitative 
assessment tools in a supported employment situation found that: (1) 
no job situation existed which required clients to demonstrate many 
of the skills or behaviors listed on the assessments; (2) some 
skills/behaviors listed on the assessments were irrelevant to the job 
situation; and (3) other relevant skills/behaviors were not listed. 
The second purpose of assessments, the measurement of program 
effectiveness, is only appropriate for groups of subjects and does 
not appropriately measure program effectiveness for individuals. Nor 
is the use of single subject designs appropriate, as they call for 
treatments that are introduced either systematically or randomly. A 
qualitative process model is suggested, where the actual process of 
implementing the model reveals the relevant variables of the job 
situation. Items in a qualitative assessment are not quant? -'^ied but 
serve as reminders of important factors to be considered. (JDD) 
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Abstract 



The utility of quantitative approaches (i.e. -assessment 
measurements) for facilitating employment for persons with 
developmental disabilities (DD) was explored. Examples were 
provided using an individual's data from an employment program 
which incorporated the use of assessments. Assessments were 
concluded to be problematic in terms of ; (1) identifying skills 
or behaviors which will facilitate an individual's employment 
and, (2) measuring an individual's progress or program 
effectiveness. A qualitative process model is proposed as a 
viable alternative. 
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Examining Crjmmiinity Employment Programs 
for Persons with MentalRetardation: 
A Comparison of Quantitative and Qualitative Research Approaches 

The recent trend toward moving vocational training for 
persons with developmental disabilities (DD) into community based 
employment settings, has encouraged a substantial amount of 
literature recommending various implementation strategies for 
such programs (Pankowsi & Rice, 1985; Wehman & Kregel, 1985; 
Rusch, Martin & White, 1985). Strategies for implementing 
community-based employment programs are intended to bridge the 
gap between an individual's attributes and the demands of their 
work environment. To facilitate this, investigators have 
developed a proliferation of assessmen intended for use in the 
implementation and evaluation of such programs (Malgady, Barcher, 
Davis & Towner, 1980; McDaniel & Flippo, 1986; Moon, Goodall, 
Barcus & Brooke, 1985) . These assessments are usually intended 
to serve one of two purposes : (1) to identify those skills or 
behaviors which the individual needs to develop in order to 
become competitively employed or; (2) to serve as a measure of 
the individual's progress in the program, and hence measure 
program effectiveness. 

The practical application for such assessments would 
involve practitioners assessing an individual to determine skills 
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for the practitioner to add any additional "skill requirements" 
which may not be included in the existing list of assessment 
skills, yet are pertinent to particular work environments (eg. 
Moon, Goodall, Barcus & Brooke, 1985; White & Rusch, 1983a; 
Rusch, 1986) . Although authors of such instruments recognize 
that the assessments may not contain an exhaustive list of skills 
or behaviors necessary for any work environment, they do, 
however, unrealistically assume that there are certain skills or 
behaviors universally applicable for all individuals and work 
settings. 

This tendency to attempt to develop a predetermined set of 
skills necessary for competitive employment seems to be a carry 
-over from the traditional approaches to vocational training. 
Conventional models assumed that by training individuals in an 
institutional setting according to a prerequisite set of skills, 
the individuals would become "job-ready". That model met with 
limited success (Kiernan & Stark, 1987; Rhodes, 1986; Wehman & 
Moon, 1965) . The current "supported employment" model assumes 
that individuals can be placed in the work environment prior to 
being job ready. Despite, this new approach to employment, 
professionals in the field are still attempting to determine a 
prerequisite set of skills which will ensure employment for 
individuals with dd. 

Through an example we will illustrate some of the problems 
that we experienced when using assessments to implement and 
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evaluate one of our supported employment projects. Ray, a 
client at Calgary's Vocational and Rehabilitation Research 
Institute was placed in a furniture manufacturing shop. We 
assessed Ray on the job site with the Vocational Adaptation 
Rating Scale (VARS) , (Malgady, Barcher, Davis & Towner, 1980) and 
the Community Integrative Scale(CIS) (The Vocational and 
Rehabilitation Research Institute, 1985) . We wanted to find out 
what skills or behaviors we should focus on in Ray's training 
program in order to maximize his chances of employment. The 
VARS assessment listed over one hundred maladaptive behaviors. 
These behaviors had been previously identified to jeopardize 
employment for persons with developmental disabilities. After 
finding out what behaviors threatened Ray's employment, we could 
try to minimize, and hopefully eliminate those behaviors through 
an appropriate training program. The CIS listed a variety of 
skills and behaviors assumed to be critical to an individual's 
successful employment in the community. Again, by assessing Ray 
on the CIS we hoped to determine needed skills for the 
development of an appropriate program. 

The first problem we encountered was that we could not 
evaluate Ray on many of the skills or behaviors listed on the 
assessments because there was no situation which required Ray to 
demonstrate such skills or behaviors. For instance, one of the 
skills listed on the VARS was - "fails to stand up for one's 
rights". Ray was never in a situation that required him to stand 
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up for his rights. Consequently we could not assess Ray on this 
item, hence making it difficult to determine whether or not we 
should train Ray in this particular area. 

Another problem we had with the assessments was that they 
failed to address some of the areas which threatened employment 
for Ray. For example, one problem for Ray was that he never 
smiled. Consequently, he appeared angry and intimidated his 
supervisors. Ray's large stature compounded the problem, when 
his supervisors would ask him if everything was alright, he 
would stare back with a very serious look on his face, and in a 
deep voice answer "yes". This response intimidatinr his 
supervisors. Despite the clear negative impact of this behavior 
regarding employability, it was not listed on any of the 
assessments we used. 

In some cases, the problem was reversed. The skills or 
behaviors listed on the assessments were irrelevant to the job 
situation. Ray scored low on several assessment items dealing 
with conversational skills including "initiating and interacting 
with others" (CIS). However, for his job this skill was not 
important. He was working alone at his work station and most of 
his co-workers were Vietnamese, who did not speak English. 

Thus, the assessments used did not provide us with the 
instrumentation needed to develop an appropriate program for Ray. 
Because we could not obtain a measure for Ray on several skills 
and, neither the list of adaptive skills nor the list of 
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maladaptive behaviors were complete; nor were all items of the 
instruments relevant for Ray's particular work environment, we 
found that the assessments were inappropriate for us. 

As mentioned at the outset, the ability to identify the 
skills necessary for competitive employment, is only one of the 
important purposes of assessments. The second purpose of the 
assessment to be considered here is the ir»easurement of program 
effectiveness. A program may be considered as a treatment and 
the effectiveness of such a treatment may be measured by first 
determining pre- treatment behavior and then post treatment 
behavior. A comparison of the two should give an indication of 
program effectiveness (Posavac & Carey, 1985) . Unfortunately, 
such designs are only appropriate for groups of subjects. Change 
in any individual may be due to random variations or the 
influence of extraneous variables. Since we wished to determine 
effectiveness for individuals such designs were not appropriate. 
Even if groups of subjects would be available, assessment 
instruments that inadequately measure the pre-treatment 
situation can not be expected to be appropriate for the 
measurement of change. 

Another strategy to measure program effectiveness may be 
through the use of single subject designs. For these designs, a 
stable baseline of behavior is established, then treatmisnts are 
introduced either systematically or randomly. A co-variation 
between treatment and absence - of - treatment measures allows 
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for an inference regarding treatment effectiveness (Conrad & 
Maul, 1981) • However,, assessment instruments of the kind 
discussed here have little utility for such designs. For such 
designs, the behavioral measure has to be relatively precise and 
appropriate to the treatment considered. Also, employment 
programs of the sort considered here do not easily lend 
themselves to the design - i,e, programs are not usually 
introduced repetitiously in a systematic or random fashion. 

We will again illustrate some of the problems we had when 
trying to evaluate Ray's progress and hence, the effectiveness of 
the program. First of all, Ray's skills and behaviors seemed to 
be quite dependent on his environment which made it difficult to 
assess whether changes in Ray's behaviors were due to the program 
or environmental circumstances. For instance one of the 
behaviors on the VARS was "displays physical and/or emotional 
outbursts, or is withdrawn". At one point in the program Ray 
became withdrawn. From talking to Ray and the prograia manager, 
we concluded that Ray^s behavior was a response to not being 
allowed to handle his own money at home. Thus the program may 
have had no bearing on this behavior. It was more likely a 
response to a change in his home environment. 

In addition, as we have seen that skills or behaviors on the 
assessments were not necessarily important tc Ray's job. *rhus, 
improvement on such skills may not mean increased likelihood of 
employment, and hence program effectiveness. For instance, if 
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Ray's convsrsational skills increased on the post assessment , Ray 
did not necessarily progress because conversational skills had 
nothing to do with continued employment. 

The message is clear • Assessnient instruments of the kind 
considered here have little utility for supported employment 
models and attempts to introduce them to the field of 
rehabilitation should be carefully evaluated. 

Each employment situation is a matter of unique and largely 
unknown person and situation characte? , sties. At this time we 
do not have enough experience nor information to allow 
predictions of success based on previous performance in various 
work situations. The reason seems to be that persons with (DD) , 
quite contrary to popular opinion, are quite adciptive to 
situations so that behaviors across situations are not constant 
and are therefore unpredictable. Therefore, serendipity rather 
than predictions and control seems to be the guiding principle. 
Perhaps with enough experience in the field, factors which are 
constant over situations and persons will emerge. These would 
then provide a starting point for a predictive model and 
assessment • 

In the meantii \, a qualitative proc*»ss Tnodel may best serve 
our purposes. The model is qualitative because the items on the 
lists are usually not quantified. Such lists are important only 
to serve ar reminders of important factors to be considered. 

The model is a process model because only the actual process 
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of implementing the model reveals the relevant variables of the 
situation. Although ^^n employer may insist that certain 
vocational skills are most important to the job, after working in 
the job setting the individual may find that other related skills 
may be crucial. Such information usually reveals itself by the 
process. 

A persons' orientation to such model should be that of an 
explorer rather than that of someone who knows the terrain. What 
is important is to gather available information, to initiate :he 
process, to be open to new information, to take time to evaluate 
what is happening, and to be ready to train, counsel or teach as 
the situation requires. 

In Ray's case it was important that problems were recognized 
as they developed, and that people and resources were available 
to deal with these problems. In our case, social skills classes 
were hy\ld weekly and resembled group counselling sessions. In 
them, the instructor made an effort to find out what was 
happening with each individual, thus addressing current problems 
and concerns as they arose. 

In conclusion, our experience has shown that assessments are 
neither necessary, sufficient, nor desirable for the supported 
employment model. We will, one more time, draw upon our 
experience with Ray to illustrate this approach. With Ray, 
assessments were not doing what we wanted because: (1) we could 
not get an accurate assessment say on all of the skills or 
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behaviors listed on the assessment, (2) the assessment 
instruments missed relevant skills and behaviors which were 
important for Ray, (3) we wasted time assessing and/or training 
him on skills or behaviors which were not important to Ray's 
employment, and (4) we were not able to measure Ray's progress in 
the program or the effectiveness of the program* 

The qualitative process model, on the other hand has several 
advantages: It's time efficient because time is not wasted 
assessing and training skills that are not important* Since it 
is time efficient it is also cost efficient* The program is 
tailored to each employment situation and recognizes that each 
individual, each employer and each set of job demands are unique. 
It is motivating for the individual because needs and demands of 
the situation are taken into consideration* Consequently, 
individuals are given more ownership over the program* 
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